key_dim is computed when splitting legacy packed in_proj_qkvz.weight but is never used afterward. Please remove it or use it for an explicit shape validation to avoid dead code.

In _prefill_with_kv_cache, the early return for not self.enable_dynamic_shape and len(prompt_tokens) > 1 means the subsequent try/except RuntimeError never executes for the “static single-token shape” case described below. As written, the fallback logic is effectively dead code for the static-shape scenario; consider simplifying to a single path (either always sequential when static, or always try batched then fall back).

The except RuntimeError fallback is currently unreachable when self.enable_dynamic_shape is True (the default): the handler re-raises whenever self.enable_dynamic_shape is true, so a static exported model that throws on batched prefill (and lacks the enable_dynamic_shape metadata method) will fail instead of falling back to sequential prefill. Consider falling back based on len(prompt_tokens) > 1 (and/or the specific error) rather than the enable_dynamic_shape flag, or updating the flag when the batched call fails.

RMSNorm.forward returns output * self.weight when add_unit_offset is false, which will promote dtypes (e.g., fp16 input → fp32 output) because self.weight is fp32. In the new add_unit_offset branch you explicitly cast the scale to type_as(x), so the output dtype now depends on the flag. Consider casting self.weight (or the final product) to type_as(x) in both branches to keep output dtype consistent with the input.

Catching a bare Exception around run_method("enable_dynamic_shape") can also hide real runtime issues (e.g., model load/ABI problems) and silently keep enable_dynamic_shape=True. It would be safer to catch the specific “method missing”/runtime exceptions raised by run_method (and optionally log at debug level) so unexpected failures don’t get swallowed.

The except Exception: ... continue around get_mapped_key will silently drop any unexpected keys (including genuinely required text weights if the mapping is incomplete or the checkpoint format changes). This makes conversion failures hard to detect. Consider only ignoring a well-defined allowlist of optional prefixes (vision/MTP/etc.) and re-raising for other model.* keys, or at least logging the first few unmapped keys at warning level.

In the non-language-model key branch, _should_ignore_unmapped_key(...) currently has no effect because the code continues unconditionally whether the key is ignored or not. This makes it easy to silently drop unexpected checkpoint keys. Consider either (a) raising for non-ignored keys here, or (b) removing the ignore-check entirely if the intent is to ignore all non-text keys.

The converter maps model.layers.*.linear_attn.conv1d.bias to layers.*.attention.conv1d.bias, but the corresponding ExecuTorch module (AttentionGatedDeltaNet) constructs conv1d with bias=False (no conv1d.bias parameter). If HF checkpoints can include this bias, it would be better to ignore/drop it during conversion (or flip the module to bias=True) to avoid carrying an unused tensor and relying on strict=False loads.

The legacy packed-key handling (in_proj_qkvz / in_proj_ba) introduces non-trivial splitting logic that isn't covered by the new unit tests. Add a test case that includes a packed key plus the required out_proj.weight so the split shapes/keys (and error paths) are exercised.

InitializedMutableBufferPass causes matched mutated buffers to be serialized with their initial values. Initializing large buffers like k_cache/v_cache will therefore increase the exported .pte size (and potentially load time / memory pressure) by the full KV-cache tensor sizes. If this is expected, consider adding a config flag (or model-class-specific opt-out) so callers can choose determinism vs artifact size, and document the expected size impact for Qwen3.5 exports.

Initializing KV cache buffers via InitializedMutableBufferPass will cause their full tensor contents to be serialized into the .pte (the emitter treats et_init_buffer+mutable_buffer as const). For k_cache/v_cache this can be extremely large (per-layer [B, H, S, D]) and may blow up export size and load time. Consider avoiding initializing the full KV caches at export (e.g., only init the small state buffers like conv_state/recurrent_state, or add a runtime/cache-reset path that deterministically zeros these buffers without serializing them).

This is a good point. @Phineas1500 does qwen3.5 require initial state for the kv-cache, conv_state and recurrent_state?

The InitializedMutableBufferPass is only required for mutable buffers with initial state.

Seems like ~5mb size increase from including initial state. Not too sure why - was expecting a bit more.

-rw-r--r-- 1 lfq users 4032780800 Mar  6 14:29 qwen3_5_0_8b_fp32_no_init.pte
-rw-r--r-- 1 lfq users 4038122240 Mar  5 11:05 qwen3_5_0_8b_fp32.pte

Output is the same with temp=0

(executorch) [lfq@devvm311.ldc0 /data/users/lfq/executorch (qwen3_5_phase2)]$ python -m executorch.examples
.models.llama.runner.native --model qwen3_5_0_8b --pte qwen3_5_0_8b_fp32_no_init.pte --tokenizer ~/.cache/h
uggingface/hub/models--Qwen--Qwen3.5-0.8B/snapshots/2fc06364715b967f1860aea9cf38778875588b17/tokenizer.json
 --tokenizer_config ~/.cache/huggingface/hub/models--Qwen--Qwen3.5-0.8B/snapshots/2fc06364715b967f1860aea9c
f38778875588b17/tokenizer_config.json --params examples/models/qwen3_5/config/0_8b_config.json --prompt "<|
im_start|>user\nHello, what's 15% of 80?<|im_end|>\n<|im_start|>assistant\n" --max_len 128 -kv --temperatur
e 0
I tokenizers:regex.cpp:27] Registering override fallback regex
Warning - given vocab_size in params is unequal to tokenizer vocab size.
[cpuinfo_utils.cpp:71] Reading file /sys/devices/soc0/image_version
[cpuinfo_utils.cpp:87] Failed to open midr file /sys/devices/soc0/image_version
[cpuinfo_utils.cpp:100] Reading file /sys/devices/system/cpu/cpu0/regs/identification/midr_el1
[cpuinfo_utils.cpp:109] Failed to open midr file /sys/devices/system/cpu/cpu0/regs/identification/midr_el1
[cpuinfo_utils.cpp:125] CPU info and manual query on # of cpus dont match.
<think>

</think>

To find 15% of 80, you can multiply 80 by 0.15:

$$80 \times 0.15 = 12$$

So, **15% of 80 is 12**.

Prefill time: 15.471495151519775
Generation tok/s: 2.097784149345492
Response: [248068, 271, 248069, 271, 1206, 1423, 220, 16, 20, 4, 314, 220, 23, 15, 11, 488, 628, 29283, 220, 23, 15, 539, 220, 15, 13, 16, 20, 25, 271, 13682, 23, 15, 1088, 14695, 220, 15, 13, 16, 20, 283, 220, 16, 17, 13682, 271, 4272, 11, 2972, 16, 20, 4, 314, 220, 23, 15, 369, 220, 16, 17, 159034, 248046]

Seems like the state is already zeroed here?
https://github.com/pytorch/executorch/blob/main/examples/models/llama/attention.py#L720

InitializedMutableBufferPass matches patterns by substring. Using "k_cache"/"v_cache" here will also match other buffer names like "k_cache_scales", "k_cache_zero_points", or "past_k_caches_*" if present in the exported graph, potentially initializing/serializing more (large) buffers than intended. If you only mean the primary caches, consider narrowing the patterns to something less collision-prone (e.g., include a delimiter or full buffer name) or splitting by known FQN fragments.